The Annotators ’ Perspective on Co - authoring with Structured Annotations

In asynchronous collaborative writing, annotations play an important roleas a communication medium among co-authors. Research has shown thatgrouping related annotations together can help those who review an anno-tated document by reducing their workload and raising the accuracy of theirreviewing. Less is known about the impact on users who create such struc-tured annotations — the annotators. The research reported in this thesishad three goals: (1) to better understand current annotation creation prac-tices, (2) to explore how structuring would be used by annotators, both thestructuring process and the resulting types of structure, and (3) to evaluatethe impact on annotators of having to create structured annotations. Weconducted three studies to address each of these goals in turn. The firststudy was an observational study which strengthened our understanding ofhow annotators use existing tools to communicate document-related infor-mation in the form of annotations. That study revealed annotation practicesthat could benefit from additional structuring support, such as annotatorsdescribing how annotations in a document relate to each other. Our secondstudy used a paper prototype system that supported annotation grouping toinvestigate how annotators would structure annotations, if given the option.Common behaviour that emerged was the grouping of thematically relatedannotations, as well as the grouping of annotations specifically targeted to agiven co-author. The study also uncovered a range of temporal approaches tostructuring annotations, such as top-down and bottom-up grouping. Basedon the first two studies, we incorporated a light-weight implicit structuringapproach based on tagging into our annotation model and then implementedan extended version of a high-fidelity prototype that supports structured an-notation, including tagging. We used the prototype in our third study, acontrolled experiment, which compared the impact of structured annotationsrelative to unstructured annotations. Participants in that study perceivedstructured annotations to be worth the additional workload required. Thestudy further suggested that the bottom-up grouping approach complementsthe top-down approach in describing relationships amongst annotations ina document.